Ten-year Mortality, Disease Progression, and Treatment-related Side Effects in Men with Localised Prostate Cancer from the ProtecT Randomised Controlled Trial According to Treatment Received.

BACKGROUND: The ProtecT trial reported intention-to-treat analysis of men with localised prostate cancer (PCa) randomly allocated to active monitoring (AM), radical prostatectomy, and external beam radiotherapy. OBJECTIVE: To determine report outcomes according to treatment received in men in randomised and treatment choice cohorts. DESIGN, SETTING, AND PARTICIPANTS: This study focuses on secondary care. Men with clinically localised prostate cancer at one of nine UK centres were invited to participate in the treatment trial comparing AM, radical prostatectomy, and radiotherapy. INTERVENTION: Two cohorts included 1643 men who agreed to be randomised; 997 declined randomisation and chose treatment. OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS: Health-related quality of life impacts on urinary, bowel, and sexual function were assessed using patient-reported outcome measures. Analysis was carried out based on treatment received for each cohort and on pooled estimates using meta-analysis. Differences were estimated with adjustment for known prognostic factors using propensity scores. RESULTS AND LIMITATIONS: According to treatment received, more men receiving AM died of PCa (AM 1.85%, surgery 0.67%, radiotherapy 0.73%), whilst this difference remained consistent with chance in the randomised cohort (p=0.08); stronger evidence was found in the exploratory analyses (randomised plus choice cohort) when AM was compared with the combined radical treatment group (p=0.003). There was also strong evidence that metastasis (AM 5.6%, surgery 2.4%, radiotherapy 2.7%) and disease progression (AM 20.35%, surgery 5.87%, radiotherapy 6.62%) were more common in the AM group. Compared with AM, there were higher risks of sexual dysfunction (95% at 6mo) and urinary incontinence (55% at 6mo) after surgery, and of sexual dysfunction (88% at 6mo) and bowel dysfunction (5% at 6mo) after radiotherapy. The key limitations are the potential for bias when comparing groups defined by treatment received and outdating of the interventions being evaluated during the lengthy follow-up required in trials of screen-detected PCa. CONCLUSIONS: Analyses according to treatment received showed increased rates of disease-related events and lower rates of patient-reported harms in men managed by AM compared with men managed by radical treatment, and stronger evidence of greater PCa mortality in the AM group. PATIENT SUMMARY: More than 90 out of every 100 men with localised prostate cancer do not die of prostate cancer within 10yr, irrespective of whether treatment is by means of monitoring, surgery, or radiotherapy. Side effects on sexual and bladder function are much better after active monitoring, but the risks of spreading of prostate cancer are more common.

Explicit inclusion of treatment in prognostic modeling was recommended in observational and randomized settings.

OBJECTIVES: To compare different methods to handle treatment when developing a prognostic model that aims to produce accurate probabilities of the outcome of individuals if left untreated. STUDY DESIGN AND SETTING: Simulations were performed based on two normally distributed predictors, a binary outcome, and a binary treatment, mimicking a randomized trial or an observational study. Comparison was made between simply ignoring treatment (SIT), restricting the analytical data set to untreated individuals (AUT), inverse probability weighting (IPW), and explicit modeling of treatment (MT). Methods were compared in terms of predictive performance of the model and the proportion of incorrect treatment decisions. RESULTS: Omitting a genuine predictor of the outcome from the prognostic model decreased model performance, in both an observational study and a randomized trial. In randomized trials, the proportion of incorrect treatment decisions was smaller when applying AUT or MT, compared to SIT and IPW. In observational studies, MT was superior to all other methods regarding the proportion of incorrect treatment decisions. CONCLUSION: If a prognostic model aims to produce correct probabilities of the outcome in the absence of treatment, ignoring treatments that affect that outcome can lead to suboptimal model performance and incorrect treatment decisions. Explicitly, modeling treatment is recommended.

The COMET initiative database: progress and activities update (2014).

The COMET Initiative database is a repository of studies relevant to the development of core outcome sets (COS). Use of the website continues to increase, with more than 16,500 visits in 2014 (36 % increase over 2013), 12,257 unique visitors (47 % increase), 9780 new visitors (43 % increase) and a rise in the proportion of visits from outside the UK (8565 visits; 51 % of all visits). By December 2014, a total of 6588 searches had been completed, with 2383 in 2014 alone (11 % increase). The growing awareness of the need for COS is reflected in the website and database usage figures.

Making randomised trials more efficient: report of the first meeting to discuss the Trial Forge platform.

Randomised trials are at the heart of evidence-based healthcare, but the methods and infrastructure for conducting these sometimes complex studies are largely evidence free. Trial Forge ( www.trialforge.org ) is an initiative that aims to increase the evidence base for trial decision making and, in doing so, to improve trial efficiency.This paper summarises a one-day workshop held in Edinburgh on 10 July 2014 to discuss Trial Forge and how to advance this initiative. We first outline the problem of inefficiency in randomised trials and go on to describe Trial Forge. We present participants' views on the processes in the life of a randomised trial that should be covered by Trial Forge.General support existed at the workshop for the Trial Forge approach to increase the evidence base for making randomised trial decisions and for improving trial efficiency. Agreed upon key processes included choosing the right research question; logistical planning for delivery, training of staff, recruitment, and retention; data management and dissemination; and close down. The process of linking to existing initiatives where possible was considered crucial. Trial Forge will not be a guideline or a checklist but a 'go to' website for research on randomised trials methods, with a linked programme of applied methodology research, coupled to an effective evidence-dissemination process. Moreover, it will support an informal network of interested trialists who meet virtually (online) and occasionally in person to build capacity and knowledge in the design and conduct of efficient randomised trials.Some of the resources invested in randomised trials are wasted because of limited evidence upon which to base many aspects of design, conduct, analysis, and reporting of clinical trials. Trial Forge will help to address this lack of evidence.

A systematic review of the use of an expertise-based randomised controlled trial design.

BACKGROUND: Under a conventional two-arm randomised trial design, participants are allocated to an intervention and participating health professionals are expected to deliver both interventions. However, health professionals often have differing levels of expertise in a skill-based interventions such as surgery or psychotherapy. An expertise-based approach to trial design, where health professionals only deliver an intervention in which they have expertise, has been proposed as an alternative. The aim of this project was to systematically review the use of an expertise-based trial design in the medical literature. METHODS: We carried out a comprehensive search of nine databases--AMED, BIOSIS, CENTRAL, CINAHL, Cochrane Methodology Register, EMBASE, MEDLINE, Science Citation Index, and PsycINFO--from 1966 to 2012 and performed citation searches using the ISI Citation Indexes and Scopus. Studies that used an expertise-based trial design were included. Two review authors independently screened the titles and abstracts and assessed full-text reports. Data were extracted and summarised on the study characteristics, general and expertise-specific study methodology, and conduct. RESULTS: In total, 7476 titles and abstracts were identified, leading to 43 included studies (54 articles). The vast majority (88%) used a pure expertise-based design; three (7%) adopted a hybrid design, and two (5%) used a design that was unclear. Most studies compared substantially different interventions (79%). In many cases, key information relating to the expertise-based design was absent; only 12 (28%) reported criteria for delivering both interventions. Most studies recruited the target sample size or very close to it (median of 101, interquartile range of 94 to 118), although the target was reported for only 40% of studies. The proportion of participants who received the allocated intervention was high (92%, interquartile range of 82 to 99%). CONCLUSIONS: While use of an expertise-based trial design is growing, it remains uncommon. Reporting of study methodology and, particularly, expertise-related methodology was poor. Empirical evidence provided some support for purported benefits such as high levels of recruitment and compliance with allocation. An expertise-based trial design should be considered but its value seems context-specific, particularly when interventions differ substantially or interventions are typically delivered by different health professionals.

The distribution of clinical phenotypes of preterm birth syndrome: implications for prevention.

IMPORTANCE: Preterm birth has been difficult to study and prevent because of its complex syndromic nature. OBJECTIVE: To identify phenotypes of preterm delivery syndrome in the Newborn Cross-Sectional Study of the INTERGROWTH-21st Project. DESIGN, SETTING, AND PARTICIPANTS: A population-based, multiethnic, cross-sectional study conducted at 8 geographically demarcated sites in Brazil, China, India, Italy, Kenya, Oman, the United Kingdom, and the United States. A total of 60,058 births over a 12-month fixed period between April 27, 2009, and March 2, 2014. Of these, 53,871 had an ultrasonography estimate of gestational age, among which 5828 were preterm births (10.8%). Pregnancies were prospectively studied using a standardized data collection and online data management system. Newborns had anthropometric and clinical examinations using standardized methods and identical equipment and were followed up until hospital discharge. MAIN OUTCOMES AND MEASURES: The main study outcomes were clusters of preterm phenotypes and for each cluster, we analyzed signs of presentation at hospital admission, admission rates for neonatal intensive care for 7 days or more, and neonatal mortality rates. RESULTS: Twelve preterm birth clusters were identified using our conceptual framework. Eleven consisted of combinations of conditions known to be associated with preterm birth, 10 of which were dominated by a single condition. However, the most common single cluster (30.0% of the total preterm cases; n = 1747) was not associated with any severe maternal, fetal, or placental condition that was clinically detectable based on the information available; within this cluster, many cases were caregiver initiated. Only 22% (n = 1284) of all the preterm births occurred spontaneously without any of these severe conditions. Maternal presentation on hospital admission, newborn anthropometry, and risk for death before hospital discharge or admission for 7 or more days to a neonatal intensive care unit, none of which were used to construct the clusters, also differed according to the identified phenotypes. The prevalence of preterm birth ranged from 8.2% in Muscat, Oman, and Oxford, England, to 16.6% in Seattle, Washington. CONCLUSIONS AND RELEVANCE: We identified 12 preterm birth phenotypes associated with different patterns of neonatal outcomes. In 22% of all preterm births, parturition started spontaneously and was not associated with any of the phenotypic conditions considered. We believe these results contribute to an improved understanding of this complex syndrome and provide an empirical basis to focus research on a more homogenous set of phenotypes.

Exploration of analysis methods for diagnostic imaging tests: problems with ROC AUC and confidence scores in CT colonography.

BACKGROUND: Different methods of evaluating diagnostic performance when comparing diagnostic tests may lead to different results. We compared two such approaches, sensitivity and specificity with area under the Receiver Operating Characteristic Curve (ROC AUC) for the evaluation of CT colonography for the detection of polyps, either with or without computer assisted detection. METHODS: In a multireader multicase study of 10 readers and 107 cases we compared sensitivity and specificity, using radiological reporting of the presence or absence of polyps, to ROC AUC calculated from confidence scores concerning the presence of polyps. Both methods were assessed against a reference standard. Here we focus on five readers, selected to illustrate issues in design and analysis. We compared diagnostic measures within readers, showing that differences in results are due to statistical methods. RESULTS: Reader performance varied widely depending on whether sensitivity and specificity or ROC AUC was used. There were problems using confidence scores; in assigning scores to all cases; in use of zero scores when no polyps were identified; the bimodal non-normal distribution of scores; fitting ROC curves due to extrapolation beyond the study data; and the undue influence of a few false positive results. Variation due to use of different ROC methods exceeded differences between test results for ROC AUC. CONCLUSIONS: The confidence scores recorded in our study violated many assumptions of ROC AUC methods, rendering these methods inappropriate. The problems we identified will apply to other detection studies using confidence scores. We found sensitivity and specificity were a more reliable and clinically appropriate method to compare diagnostic tests.

International standards for newborn weight, length, and head circumference by gestational age and sex: the Newborn Cross-Sectional Study of the INTERGROWTH-21st Project.

BACKGROUND: In 2006, WHO published international growth standards for children younger than 5 years, which are now accepted worldwide. In the INTERGROWTH-21(st) Project, our aim was to complement them by developing international standards for fetuses, newborn infants, and the postnatal growth period of preterm infants. METHODS: INTERGROWTH-21(st) is a population-based project that assessed fetal growth and newborn size in eight geographically defined urban populations. These groups were selected because most of the health and nutrition needs of mothers were met, adequate antenatal care was provided, and there were no major environmental constraints on growth. As part of the Newborn Cross-Sectional Study (NCSS), a component of INTERGROWTH-21(st) Project, we measured weight, length, and head circumference in all newborn infants, in addition to collecting data prospectively for pregnancy and the perinatal period. To construct the newborn standards, we selected all pregnancies in women meeting (in addition to the underlying population characteristics) strict individual eligibility criteria for a population at low risk of impaired fetal growth (labelled the NCSS prescriptive subpopulation). Women had a reliable ultrasound estimate of gestational age using crown-rump length before 14 weeks of gestation or biparietal diameter if antenatal care started between 14 weeks and 24 weeks or less of gestation. Newborn anthropometric measures were obtained within 12 h of birth by identically trained anthropometric teams using the same equipment at all sites. Fractional polynomials assuming a skewed t distribution were used to estimate the fitted centiles. FINDINGS: We identified 20,486 (35%) eligible women from the 59,137 pregnant women enrolled in NCSS between May 14, 2009, and Aug 2, 2013. We calculated sex-specific observed and smoothed centiles for weight, length, and head circumference for gestational age at birth. The observed and smoothed centiles were almost identical. We present the 3rd, 10th, 50th, 90th, and 97th centile curves according to gestational age and sex. INTERPRETATION: We have developed, for routine clinical practice, international anthropometric standards to assess newborn size that are intended to complement the WHO Child Growth Standards and allow comparisons across multiethnic populations. FUNDING: Bill & Melinda Gates Foundation.

Coronary artery bypass grafting in high-RISk patients randomised to off- or on-Pump surgery: a randomised controlled trial (the CRISP trial).

BACKGROUND: Coronary artery bypass grafting (CABG) is the treatment of choice for patients with multivessel coronary artery disease (CAD). Evidence from randomised controlled trials (RCTs) in low-risk populations shows that 'off-pump' CABG is at least as safe as 'on-pump' CABG, but high-quality trial data in high-risk populations are lacking. OBJECTIVES: To test the hypothesis that, in high-risk patients, off-pump coronary artery bypass grafting (OPCABG) reduces mortality and morbidity without causing a higher risk of reintervention compared with on-pump coronary artery bypass grafting (ONCABG). DESIGN: Open parallel-group RCT with a 1 : 1 allocation ratio and expertise-based randomisation. SETTING: Eight specialist cardiac surgery centres in the UK and one specialist centre in Kolkata, India. PARTICIPANTS: Patients with an additive European system for cardiac operative risk evaluation score (EuroSCORE) of ≥ 5, undergoing non-emergency isolated CABG via a median sternotomy. INTERVENTIONS: CABG without cardiopulmonary bypass (CPB), i.e. OPCABG on the beating heart, or CABG with CPB, i.e. ONCABG on a chemically arrested heart. MAIN OUTCOME MEASURES: Primary outcome - a composite of death or serious morbidity [all-cause mortality, myocardial infarction (MI), stroke, prolonged initial ventilation, sternal wound dehiscence] within 30 days of surgery. Secondary outcomes - quality of life (QoL) [Rose Angina Questionnaire, Canadian Cardiovascular Society (CCS) angina class, European QoL-5 Dimensions (EQ-5D), Coronary Revascularisation Outcome Questionnaire (CROQ)] and resource utilisation. RESULTS: The organisation of a tertiary cardiac surgery service in the UK presented several barriers to recruitment. Referral information was often inadequate to confirm eligibility. Limited surgeon participation at a centre, the need to meet referral-to-treatment performance targets and complex referral pathways did not support an expertise-based allocation. Urgent patients waiting for surgery in local 'feeder' hospitals were often not transferred until late the night before surgery, which limited the time available to take consent and organise the surgery on an expertise basis. Several elective patients declined to take part because they wanted the surgeon they had met when the surgery was first discussed in clinic to operate. Several initiatives were explored to boost recruitment. After 10 months of recruitment, the trial design was modified to permit both within-surgeon and expertise-based randomisation within a centre. However, this did not have sufficient impact and the trial was stopped on the grounds of futility after 106 patients (< 2% of the target sample size) had been recruited in 18 months. Ninety-eight patients were included in the trial analyses, six patients were withdrawn and two died before surgery. In both groups, 6% of patients experienced the primary outcome [adjusted odds ratio (OR) (OPCABG to ONCABG) 1.07; 95% confidence interval (CI) 0.27 to 4.14]. QoL scores at 4-8 weeks post surgery were similar in the two groups. Patients randomised to OPCABG had a shorter stay in the intensive care unit and in hospital after surgery (median 26.0 vs. 27.7 hours in intensive care and 7 vs. 8 days in hospital). CONCLUSIONS: The Coronary artery bypass grafting in high-RISk patients randomised to off- or on-Pump surgery (CRISP) trial was not successful for a range of logistical reasons. However, the experience gained is of value for the design and conduct of future trials. The surgical community have polarised views. A qualitative evaluation of the reasons behind the views held by the advocates of the two techniques is an area for future research. TRIAL REGISTRATION: Current Controlled Trials ISRCTN29161170. FUNDING: This project was funded by the Medical Research Council/National Institute for Health Research (NIHR) Efficacy and Mechanism Evaluation programme and will be published in full in Health Technology Assessment; Vol. 18, No. 44. See the NIHR Journals Library website for further project information.

The likeness of fetal growth and newborn size across non-isolated populations in the INTERGROWTH-21st Project: the Fetal Growth Longitudinal Study and Newborn Cross-Sectional Study.

BACKGROUND: Large differences exist in size at birth and in rates of impaired fetal growth worldwide. The relative effects of nutrition, disease, the environment, and genetics on these differences are often debated. In clinical practice, various references are often used to assess fetal growth and newborn size across populations and ethnic origins, whereas international standards for assessing growth in infants and children have been established. In the INTERGROWTH-21(st) Project, our aim was to assess fetal growth and newborn size in eight geographically defined urban populations in which the health and nutrition needs of mothers were met and adequate antenatal care was provided. METHODS: For this study, fetal growth and newborn size were measured in two INTERGROWTH-21(st) component studies using prespecified markers and the same methods, equipment, and selection criteria. In the Fetal Growth Longitudinal Study (FGLS), we studied educated, affluent, healthy women, with adequate nutritional status who were at low risk of intrauterine growth restriction. The primary markers of fetal growth were ultrasound measurements of fetal crown-rump length at less than 14 weeks and 0 days of gestation and fetal head circumference from 14 weeks and 0 days to 40 weeks and 0 days of gestation, and birthlength for newborn size. In the concomitant, population-based Newborn Cross-Sectional Study (NCSS), we measured birthlength in all newborn babies from the eight geographically defined urban populations with the same methods, instruments, and staff as in FGLS. From this large NCSS cohort, we selected an FGLS-like subpopulation to match FGLS with the same eligibility criteria. FINDINGS: Between May 14, 2009, and Aug 2, 2013, we enrolled 4607 women in FGLS and 59 137 women in NCSS. From NCSS, 20 486 (34·6%) women met the FGLS eligibility criteria, and constituted the FGLS-like subpopulation. With variance component analysis, only between 1·9% and 3·5% of the total variability in crown-rump length, fetal head circumference, and newborn birthlength could be attributed to between-site differences. With standardised site effect analysis in 16 gestational age windows from 9 weeks and 0 days of gestation to birth for the three measures (128 comparisons), only one was marginally higher than 0·5 SD of the standardised site difference range. Sensitivity analyses, excluding individual populations in turn from the pooling of all-site centiles across gestational ages, showed no noticeable effect on the 3rd, 50th, and 97th centiles derived from the remaining populations. Our populations were consistent at birth with those in the WHO Multicentre Growth Reference Study (MGRS). The mean birthlength for term newborn babies in that study was 49·5 cm (SD 1·9), which was very similar to that in the FGLS cohort (49·4 cm [1·9]) and the NCSS derived FGLS-like subpopulation (49·3 cm [1·8]). INTERPRETATION: Fetal growth and newborn length are similar across diverse geographical settings when mothers' nutritional and health needs are met, and environmental constraints on growth are low. The findings for birthlength are in strong agreement with those of the WHO MGRS. These results provide the conceptual frame to create international standards for growth from conception to newborn baby, which will extend the present infant to childhood WHO MGRS standards. FUNDING: Bill & Melinda Gates Foundation.

Choosing important health outcomes for comparative effectiveness research: a systematic review.

BACKGROUND: A core outcome set (COS) is a standardised set of outcomes which should be measured and reported, as a minimum, in all effectiveness trials for a specific health area. This will allow results of studies to be compared, contrasted and combined as appropriate, as well as ensuring that all trials contribute usable information. The COMET (Core Outcome Measures for Effectiveness Trials) Initiative aims to support the development, reporting and adoption of COS. Central to this is a publically accessible online resource, populated with all available COS. The aim of the review we report here was to identify studies that sought to determine which outcomes or domains to measure in all clinical trials in a specific condition and to describe the methodological techniques used in these studies. METHODS: We developed a multi-faceted search strategy for electronic databases (MEDLINE, SCOPUS, and Cochrane Methodology Register). We included studies that sought to determine which outcomes/domains to measure in all clinical trials in a specific condition. RESULTS: A total of 250 reports relating to 198 studies were judged eligible for inclusion in the review. Studies covered various areas of health, most commonly cancer, rheumatology, neurology, heart and circulation, and dentistry and oral health. A variety of methods have been used to develop COS, including semi-structured discussion, unstructured group discussion, the Delphi Technique, Consensus Development Conference, surveys and Nominal Group Technique. The most common groups involved were clinical experts and non-clinical research experts. Thirty-one (16%) studies reported that the public had been involved in the process. The geographic locations of participants were predominantly North America (n = 164; 83%) and Europe (n = 150; 76%). CONCLUSIONS: This systematic review identified many health areas where a COS has been developed, but also highlights important gaps. It is a further step towards a comprehensive, up-to-date database of COS. In addition, it shows the need for methodological guidance, including how to engage key stakeholder groups, particularly members of the public.

Red blood cell transfusion and mortality in trauma patients: risk-stratified analysis of an observational study.

BACKGROUND: Haemorrhage is a common cause of death in trauma patients. Although transfusions are extensively used in the care of bleeding trauma patients, there is uncertainty about the balance of risks and benefits and how this balance depends on the baseline risk of death. Our objective was to evaluate the association of red blood cell (RBC) transfusion with mortality according to the predicted risk of death. METHODS AND FINDINGS: A secondary analysis of the CRASH-2 trial (which originally evaluated the effect of tranexamic acid on mortality in trauma patients) was conducted. The trial included 20,127 trauma patients with significant bleeding from 274 hospitals in 40 countries. We evaluated the association of RBC transfusion with mortality in four strata of predicted risk of death: <6%, 6%-20%, 21%-50%, and >50%. For this analysis the exposure considered was RBC transfusion, and the main outcome was death from all causes at 28 days. A total of 10,227 patients (50.8%) received at least one transfusion. We found strong evidence that the association of transfusion with all-cause mortality varied according to the predicted risk of death (p-value for interaction <0.0001). Transfusion was associated with an increase in all-cause mortality among patients with <6% and 6%-20% predicted risk of death (odds ratio [OR] 5.40, 95% CI 4.08-7.13, p<0.0001, and OR 2.31, 95% CI 1.96-2.73, p<0.0001, respectively), but with a decrease in all-cause mortality in patients with >50% predicted risk of death (OR 0.59, 95% CI 0.47-0.74, p<0.0001). Transfusion was associated with an increase in fatal and non-fatal vascular events (OR 2.58, 95% CI 2.05-3.24, p<0.0001). The risk associated with RBC transfusion was significantly increased for all the predicted risk of death categories, but the relative increase was higher for those with the lowest (<6%) predicted risk of death (p-value for interaction <0.0001). As this was an observational study, the results could have been affected by different types of confounding. In addition, we could not consider haemoglobin in our analysis. In sensitivity analyses, excluding patients who died early; conducting propensity score analysis adjusting by use of platelets, fresh frozen plasma, and cryoprecipitate; and adjusting for country produced results that were similar. CONCLUSIONS: The association of transfusion with all-cause mortality appears to vary according to the predicted risk of death. Transfusion may reduce mortality in patients at high risk of death but increase mortality in those at low risk. The effect of transfusion in low-risk patients should be further tested in a randomised trial. TRIAL REGISTRATION: www.ClinicalTrials.gov NCT01746953.

Use of methods for specifying the target difference in randomised controlled trial sample size calculations: Two surveys of trialists' practice.

BACKGROUND: Central to the design of a randomised controlled trial (RCT) is a calculation of the number of participants needed. This is typically achieved by specifying a target difference, which enables the trial to identify a difference of a particular magnitude should one exist. Seven methods have been proposed for formally determining what the target difference should be. However, in practice, it may be driven by convenience or some other informal basis. It is unclear how aware the trialist community is of these formal methods or whether they are used. PURPOSE: To determine current practice regarding the specification of the target difference by surveying trialists. METHODS: Two surveys were conducted: (1) Members of the Society for Clinical Trials (SCT): participants were invited to complete an online survey through the society's email distribution list. Respondents were asked about their awareness, use of, and willingness to recommend methods; (2) Leading UK- and Ireland-based trialists: the survey was sent to UK Clinical Research Collaboration registered Clinical Trials Units, Medical Research Council UK Hubs for Trial Methodology Research, and the Research Design Services of the National Institute for Health Research. This survey also included questions about the most recent trial developed by the respondent's group. RESULTS: Survey 1: Of the 1182 members on the SCT membership email distribution list, 180 responses were received (15%). Awareness of methods ranged from 69 (38%) for health economic methods to 162 (90%) for pilot study. Willingness to recommend among those who had used a particular method ranged from 56% for the opinion-seeking method to 89% for the review of evidence-base method. Survey 2: Of the 61 surveys sent out, 34 (56%) responses were received. Awareness of methods ranged from 33 (97%) for the review of evidence-base and pilot methods to 14 (41%) for the distribution method. The highest level of willingness to recommend among users was for the anchor method (87%). Based upon the most recent trial, the target difference was usually one viewed as important by a stakeholder group, mostly also viewed as a realistic difference given the interventions under evaluation, and sometimes one that led to an achievable sample size. LIMITATIONS: The response rates achieved were relatively low despite the surveys being short, well presented, and having utilised reminders. CONCLUSION: Substantial variations in practice exist with awareness, use, and willingness to recommend methods varying substantially. The findings support the view that sample size calculation is a more complex process than would appear to be the case from trial reports and protocols. Guidance on approaches for sample size estimation may increase both awareness and use of appropriate formal methods.

Consensus-based recommendations for investigating clinical heterogeneity in systematic reviews.

BACKGROUND: Critics of systematic reviews have argued that these studies often fail to inform clinical decision making because their results are far too general, that the data are sparse, such that findings cannot be applied to individual patients or for other decision making. While there is some consensus on methods for investigating statistical and methodological heterogeneity, little attention has been paid to clinical aspects of heterogeneity. Clinical heterogeneity, true effect heterogeneity, can be defined as variability among studies in the participants, the types or timing of outcome measurements, and the intervention characteristics. The objective of this project was to develop recommendations for investigating clinical heterogeneity in systematic reviews. METHODS: We used a modified Delphi technique with three phases: (1) pre-meeting item generation; (2) face-to-face consensus meeting in the form of a modified Delphi process; and (3) post-meeting feedback. We identified and invited potential participants with expertise in systematic review methodology, systematic review reporting, or statistical aspects of meta-analyses, or those who published papers on clinical heterogeneity. RESULTS: Between April and June of 2011, we conducted phone calls with participants. In June 2011 we held the face-to-face focus group meeting in Ann Arbor, Michigan. First, we agreed upon a definition of clinical heterogeneity: Variations in the treatment effect that are due to differences in clinically related characteristics. Next, we discussed and generated recommendations in the following 12 categories related to investigating clinical heterogeneity: the systematic review team, planning investigations, rationale for choice of variables, types of clinical variables, the role of statistical heterogeneity, the use of plotting and visual aids, dealing with outlier studies, the number of investigations or variables, the role of the best evidence synthesis, types of statistical methods, the interpretation of findings, and reporting. CONCLUSIONS: Clinical heterogeneity is common in systematic reviews. Our recommendations can help guide systematic reviewers in conducting valid and reliable investigations of clinical heterogeneity. Findings of these investigations may allow for increased applicability of findings of systematic reviews to the management of individual patients.

Inconsistency between direct and indirect comparisons of competing interventions: meta-epidemiological study.

OBJECTIVE: To investigate the agreement between direct and indirect comparisons of competing healthcare interventions. DESIGN: Meta-epidemiological study based on sample of meta-analyses of randomised controlled trials. Data sources Cochrane Database of Systematic Reviews and PubMed. Inclusion criteria Systematic reviews that provided sufficient data for both direct comparison and independent indirect comparisons of two interventions on the basis of a common comparator and in which the odds ratio could be used as the outcome statistic. MAIN OUTCOME MEASURE: Inconsistency measured by the difference in the log odds ratio between the direct and indirect methods. RESULTS: The study included 112 independent trial networks (including 1552 trials with 478,775 patients in total) that allowed both direct and indirect comparison of two interventions. Indirect comparison had already been explicitly done in only 13 of the 85 Cochrane reviews included. The inconsistency between the direct and indirect comparison was statistically significant in 16 cases (14%, 95% confidence interval 9% to 22%). The statistically significant inconsistency was associated with fewer trials, subjectively assessed outcomes, and statistically significant effects of treatment in either direct or indirect comparisons. Owing to considerable inconsistency, many (14/39) of the statistically significant effects by direct comparison became non-significant when the direct and indirect estimates were combined. CONCLUSIONS: Significant inconsistency between direct and indirect comparisons may be more prevalent than previously observed. Direct and indirect estimates should be combined in mixed treatment comparisons only after adequate assessment of the consistency of the evidence.

Assessing and reporting heterogeneity in treatment effects in clinical trials: a proposal.

Mounting evidence suggests that there is frequently considerable variation in the risk of the outcome of interest in clinical trial populations. These differences in risk will often cause clinically important heterogeneity in treatment effects (HTE) across the trial population, such that the balance between treatment risks and benefits may differ substantially between large identifiable patient subgroups; the "average" benefit observed in the summary result may even be non-representative of the treatment effect for a typical patient in the trial. Conventional subgroup analyses, which examine whether specific patient characteristics modify the effects of treatment, are usually unable to detect even large variations in treatment benefit (and harm) across risk groups because they do not account for the fact that patients have multiple characteristics simultaneously that affect the likelihood of treatment benefit. Based upon recent evidence on optimal statistical approaches to assessing HTE, we propose a framework that prioritizes the analysis and reporting of multivariate risk-based HTE and suggests that other subgroup analyses should be explicitly labeled either as primary subgroup analyses (well-motivated by prior evidence and intended to produce clinically actionable results) or secondary (exploratory) subgroup analyses (performed to inform future research). A standardized and transparent approach to HTE assessment and reporting could substantially improve clinical trial utility and interpretability.

Bias due to changes in specified outcomes during the systematic review process.

BACKGROUND: Adding, omitting or changing outcomes after a systematic review protocol is published can result in bias because it increases the potential for unacknowledged or post hoc revisions of the planned analyses. The main objective of this study was to look for discrepancies between primary outcomes listed in protocols and in the subsequent completed reviews published on the Cochrane Library. A secondary objective was to quantify the risk of bias in a set of meta-analyses where discrepancies between outcome specifications in protocols and reviews were found. METHODS AND FINDINGS: New reviews from three consecutive issues of the Cochrane Library were assessed. For each review, the primary outcome(s) listed in the review protocol and the review itself were identified and review authors were contacted to provide reasons for any discrepancies. Over a fifth (64/288, 22%) of protocol/review pairings were found to contain a discrepancy in at least one outcome measure, of which 48 (75%) were attributable to changes in the primary outcome measure. Where lead authors could recall a reason for the discrepancy in the primary outcome, there was found to be potential bias in nearly a third (8/28, 29%) of these reviews, with changes being made after knowledge of the results from individual trials. Only 4(6%) of the 64 reviews with an outcome discrepancy described the reason for the change in the review, with no acknowledgment of the change in any of the eight reviews containing potentially biased discrepancies. Outcomes that were promoted in the review were more likely to be significant than if there was no discrepancy (relative risk 1.66 95% CI (1.10, 2.49), p = 0.02). CONCLUSION: In a review, making changes after seeing the results for included studies can lead to biased and misleading interpretation if the importance of the outcome (primary or secondary) is changed on the basis of those results. Our assessment showed that reasons for discrepancies with the protocol are not reported in the review, demonstrating an under-recognition of the problem. Complete transparency in the reporting of changes in outcome specification is vital; systematic reviewers should ensure that any legitimate changes to outcome specification are reported with reason in the review.

Evaluation of the Ages and Stages Questionnaires in identifying children with neurosensory disability in the Magpie Trial follow-up study.

AIM: To evaluate performance of the Ages and Stages Questionnaires (full ASQ), and a shortened version (short ASQ), in detecting children with severe neurosensory disability in the Magpie Trial follow-up study. METHODS: All children, born to women in the Magpie Trial and selected for follow-up, with a completed full 30 items and/or short 9-items ASQ were included in this analysis. Sensitivity and specificity, corrected for verification bias, were computed to assess detection ability. RESULTS: Of the 2046 children who completed a full ASQ, 406 (19.8%) failed the assessment, 54 of whom had confirmed neurosensory disability. Adjusted sensitivity and specificity (95% confidence intervals) were 87.4% (62.9-96.6%), and 82.3% (80.5-83.9%), respectively. Two of the five domains in the full ASQ (Fine Motor and Problem Solving) contributed little to detection ability. Sensitivity and specificity for the short ASQ were 69.2% and 95.7%, respectively. CONCLUSIONS: Sensitivity of the full ASQ for severe neurosensory disability is generally good, and does not appear to be much reduced by restricting questions to three out of the five domains. The short ASQ reported here reduced performance, although this might be improved by a different choice of questions or scoring system.